Question 1 Individual

Ratio

Q 1a

p value
p1 0.5
p2 0.3
p3 0.2

Q 1b

p value
p1 0.50
p2 0.48
p3 0.02

Row

1a Method 1

1b Method 1

Row

1a Method 2

1b Method 2

Question 1 Joint

Ratio

Q 1a

p value
p1 0.5
p2 0.3
p3 0.2

Q 1b

p value
p1 0.50
p2 0.48
p3 0.02

Column

Question 1a

Column

Question 1b

Question 2 Individual

Ratio

p value
p1 0.46
p2 0.44
p3 0.10
r value
r1 0.34
r2 0.33
r3 0.33
delta value
delta 1 0.02
delta 2 0.36
delta 3 0.34

Row

Row

Question 2 Joint

Ratio

p value
p1 0.46
p2 0.44
p3 0.10
r value
r1 0.34
r2 0.33
r3 0.33
delta value
delta 1 0.02
delta 2 0.36
delta 3 0.34

Column

Method 1

Column

Method 2

Summary

Column

Summary

Goal: Construct simultaneous 95% confidence intervals

1. Without contamination

Method 1:

  • The coverage ratios are similar to each individual pair difference. And they are way over 95%.
  • The joint coverage ratio is also much higher than the target 95%.
  • Too conservative. Increasing sample size is making it even more conservative.

Method 2:

  • The coverage ratios are not constant over all pair differences. One of the three pair difference would have distinct lower coverage ratio than others.
  • That relatively low coverage pair difference caps the joint coverage ratio, leading to the desired 95%.
  • Relatively exact. Increasing sample may lower the ratio to target.

2. With contamination

Individual Coverage:

Both methods show similiar patterns:

The more pair difference diverage to its original value, the more it tends to be affected by the mix ratio.

  • true value 0.02: almost same before and after contamination -> limited impact
  • true value 0.36: decreasing to 0.01 (completely replaced) -> most serious impact

Joint Coverage

Overall, method 2 has better performance than method 1.

Column

Joint coverage for data

sampleSize question method_1 method_2 close2target
500 1a 0.9780 0.9595 method 2
1000 1a 0.9830 0.9655 method 2
1500 1a 0.9835 0.9645 method 2
500 1b 0.9890 0.9610 method 2
1000 1b 0.9895 0.9565 method 2
1500 1b 0.9890 0.9525 method 2

Joint coverage for mixed data

sampleSize mixRatio method_1 method_2 higherCoverRatio
500 0.1 0.9215 0.9205 method 1
500 0.2 0.5730 0.5910 method 2
500 0.3 0.1420 0.1475 method 2
500 0.4 0.0135 0.0140 method 2
500 0.5 0.0005 0.0005 method 2
1000 0.1 0.8080 0.8360 method 2
1000 0.2 0.1625 0.1865 method 2
1000 0.3 0.0020 0.0025 method 2
1000 0.4 0.0000 0.0000 method 2
1000 0.5 0.0000 0.0000 method 2
1500 0.1 0.6715 0.7180 method 2
1500 0.2 0.0335 0.0415 method 2
1500 0.3 0.0000 0.0000 method 2
1500 0.4 0.0000 0.0000 method 2
1500 0.5 0.0000 0.0000 method 2